Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(http): support for FormData requests #6708

Merged
merged 5 commits into from Jul 11, 2023

Conversation

kjr-lh
Copy link
Contributor

@kjr-lh kjr-lh commented Jul 5, 2023

This is #6206 for capacitor 5 (tested against 5.0.5)

Also includes a fix for the content-type bug mentioned here

It wasn't checking for lower case, resulting in json being passed back instead of a string
@kjr-lh kjr-lh mentioned this pull request Jul 5, 2023
4 tasks
@kjr-lh kjr-lh changed the title Http type fixes for capacitor 5 Http formdata and file support for capacitor 5 Jul 5, 2023
Copy link
Contributor

@markemer markemer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks the same as #6206. Needs to be updated to merge.

@ItsChaceD ItsChaceD changed the title Http formdata and file support for capacitor 5 feat(http): support for FormData requests Jul 7, 2023
@markemer markemer merged commit 849c564 into ionic-team:main Jul 11, 2023
6 checks passed
@jreviews
Copy link

Thanks for working on this! Would it be possible to see an example of how sending formdata in a post request would work? I've been trying all sorts of things and passing a FormData object with some appended data never makes it to the server. If I use a json object then it works fine. I tried setting the dataType to formData and that also didn't help. My goal is to eventually send a file selected from Camera.photos, but for not just trying with any simple data.

@mikeelemuel
Copy link

mikeelemuel commented Jul 17, 2023

@kjr-lh , I tried to update to 5.2.1 version which this version includes your changes.

However, when I tried the code below, I'm getting this on xcode terminal [NSURLSession sharedSession] may not be invalidated. Question, did I miss something based on your changes?

       const applicationAdapter = this.store.adapterFor('application');
       const headers            = {
          'X-APP-REF':     applicationAdapter.headers['X-APP-REF'],
          'X-APP-VERSION': applicationAdapter.headers['X-APP-VERSION'],
          'Authorization': applicationAdapter.headers['Authorization'],
       }
         
        const formData = new FormData();
        formData.append('visibility', this.visibility);
        formData.append('attachable_type', classify(this.args.attachableRecord.constructor.modelName));
        formData.append('attachable_id', this.args.attachableRecord.get('id'));
        formData.set('Content-Type', file.type);
        formData.append('attachment', file.file);

        await fetch(`/v3/${this.current.accountSubdomain}/attachments`, {
          method: 'POST',
          headers: headers,
          body: formData,
        });

@jreviews
Copy link

jreviews commented Jul 17, 2023

I thought that this PR would allow using CapacitorHttp to make requests with FormData. Is that not the case? I tested using fetch directly and that worked, but using CapacitorHttp doesn't. My test code to upload a photo selected from photos with fetch:

const dataURItoBlob = (dataURI: string) => {
    // Split the `dataURI` at the comma.
    let splitDataURI = dataURI.split(',');

    // Get the mime-type from the `dataURI`
    let mimeType = splitDataURI[0].split(':')[1].split(';')[0];

    // Get the base64 string
    const base64 = splitDataURI[1];
    
    // Decode the base64 string
    let binary = atob(base64);
    
    // Create an 8-bit unsigned array
    let array = [];
    for(let i = 0; i < binary.length; i++) {
        array.push(binary.charCodeAt(i));
    }
    
    // Return the blob object with appropriate mime-type
    return new Blob([new Uint8Array(array)], {type: mimeType});
}

const upload = async () => {

    const photo = await Camera.getPhoto({
            resultType: CameraResultType.DataUrl,
            source: CameraSource.Photos,
            correctOrientation: true,
            quality: 90,
            allowEditing: false
        })

    const blob = dataURItoBlob(photo.dataUrl)

    const formData = new FormData()

    formData.append('file', blob)

    fetch(useAppConfig().getApiUrl('/upload-test'), {
            method: 'POST',
            body: formData
        })
        .then(response => response.json())
        .then(result => {
            console.log('Success:', result);
        })    
}

@jreviews
Copy link

In case this is helpful for anyone else, the above code using fetch does end up triggering a native CapacitorHttp.request as seen in the console image below.

image

However, I couldn't find a way to just issue the CapacitorHttp.request directly in any way that it would work. The closest I got was still missing the boundary part of the multipart/form-data header so it failed. So for now I'll stick to what's working for me.

@muuvmuuv

This comment was marked as off-topic.

@kjr-lh
Copy link
Contributor Author

kjr-lh commented Jul 19, 2023

I thought that this PR would allow using CapacitorHttp to make requests with FormData. Is that not the case? I tested using fetch directly and that worked, but using CapacitorHttp doesn't. My test code to upload a photo selected from photos with fetch:

const dataURItoBlob = (dataURI: string) => {
    // Split the `dataURI` at the comma.
    let splitDataURI = dataURI.split(',');

    // Get the mime-type from the `dataURI`
    let mimeType = splitDataURI[0].split(':')[1].split(';')[0];

    // Get the base64 string
    const base64 = splitDataURI[1];
    
    // Decode the base64 string
    let binary = atob(base64);
    
    // Create an 8-bit unsigned array
    let array = [];
    for(let i = 0; i < binary.length; i++) {
        array.push(binary.charCodeAt(i));
    }
    
    // Return the blob object with appropriate mime-type
    return new Blob([new Uint8Array(array)], {type: mimeType});
}

const upload = async () => {

    const photo = await Camera.getPhoto({
            resultType: CameraResultType.DataUrl,
            source: CameraSource.Photos,
            correctOrientation: true,
            quality: 90,
            allowEditing: false
        })

    const blob = dataURItoBlob(photo.dataUrl)

    const formData = new FormData()

    formData.append('file', blob)

    fetch(useAppConfig().getApiUrl('/upload-test'), {
            method: 'POST',
            body: formData
        })
        .then(response => response.json())
        .then(result => {
            console.log('Success:', result);
        })    
}

This doesn't add Blob support yet - if you can pass your data as File it will work

@kjr-lh
Copy link
Contributor Author

kjr-lh commented Jul 19, 2023

[NSURLSession sharedSession] may not be invalidate

Hi @mikeelemuel, your code looks like it should be okay.. the only thing I can think to try would be to include the filename , e.g.

formData.append('attachment', file.file, 'somefilename.txt');

But I'm really doubtful this will help 😞

I'm afraid I don't have enough Swift experience to diagnose that error

@jreviews
Copy link

@kjr-lh Thanks!

This doesn't add Blob support yet - if you can pass your data as File it will work

I may try that eventually. However, removing the file from formData and just leaving a plain input with text also didn't work for me as mentioned here #6708 (comment) The data doesn't arrive to the server.

@kjr-lh
Copy link
Contributor Author

kjr-lh commented Jul 20, 2023

@jreviews I've created a demo app here that shows uploading FormData with file and text fields. I'll use this to add and test Blob support soon

@jreviews
Copy link

@kjr-lh Awesome, thanks. However, I don't see that you are using CapacitorHttp there. Per my previous comments, I have it working with fetch, but I thought using CapacitorHttp.request/post with FormData should also work and when I last tested, the data doesn't make it to the server (file or not).

@kjr-lh
Copy link
Contributor Author

kjr-lh commented Jul 21, 2023

@kjr-lh Awesome, thanks. However, I don't see that you are using CapacitorHttp there. Per my previous comments, I have it working with fetch, but I thought using CapacitorHttp.request/post with FormData should also work and when I last tested, the data doesn't make it to the server (file or not).

Fetch is capacitorhttp now (from one of the 4.x versions) - Capacitor patches fetch and XMLHttpRequest with it's own versions that go through the native layer

https://capacitorjs.com/docs/apis/http

The code that is handling Files / FormData is in the fetch/XMLHttpRequest implementations, so you'll be skipping it if you go directly to CapacitorHttp request/post. You could encode the data yourself, but it'd be much simpler to use fetch

@jreviews
Copy link

jreviews commented Jul 21, 2023

The code that is handling Files / FormData is in the fetch/XMLHttpRequest implementations

I understand that from my tests, but what was confusing to me since I was waiting for the FormData update is that there's nothing in the documentation about having to use fetch directly to be able to also use FormData/Files.

What made it even more confusing is that I went through the code changes to try to understand how to use FormData and saw this:

https://github.com/ionic-team/capacitor/pull/6708/files#diff-63eeefca71ce9525d8b4f78010d981ea646a106657519835b66b1bf47b2aac70R201

So basically a new option was added to HttpOptions (which is used with CapacitorHttp) to specify file/formData

export interface HttpOptions {
  url: string;
  method?: string;
  params?: HttpParams;
  data?: any;
  headers?: HttpHeaders;
  /**
   * How long to wait to read additional data. Resets each time new
   * data is received
   */
  readTimeout?: number;
  /**
   * How long to wait for the initial connection.
   */
  connectTimeout?: number;
  /**
   * Sets whether automatic HTTP redirects should be disabled
   */
  disableRedirects?: boolean;
  /**
   * Extra arguments for fetch when running on the web
   */
  webFetchExtra?: RequestInit;
  /**
   * This is used to parse the response appropriately before returning it to
   * the requestee. If the response content-type is "json", this value is ignored.
   */
  responseType?: HttpResponseType;
  /**
   * Use this option if you need to keep the URL unencoded in certain cases
   * (already encoded, azure/firebase testing, etc.). The default is _true_.
   */
  shouldEncodeUrlParams?: boolean;
  /**
   * This is used if we've had to convert the data from a JS type that needs
   * special handling in the native layer
   */
  dataType?: 'file' | 'formData';
}

Finally, what's even more confusing is that when using fetch with FormData, everything does work, and it ends up using CapacitorHttp under the hood per my previous image

image

Which makes me believe there must be a way to use CapacitorHttp directly to be able to use the Capacitor API rather than using fetch.

I am happy to have it working now. I am just trying to provide feedback from the point of view of someone that uses the framework and was using the community/http-plugin before to upload files which had an uploadFile method. The documentation could use a bit more information on this, similar to the FileSystem docs for downloads where I also had to go through the test app code to figure out how to migrate.

@muuvmuuv
Copy link

muuvmuuv commented Jul 21, 2023

You can @jreviews, if you mean this:

import { CapacitorHttp } from "@capacitor/core"
import { Filesystem } from "@capacitor/filesystem"

export async function fileUpload(file: {
  name: string
  path: string
  mimeType: string
}) {
  const formData = new FormData()

  const contents = await Filesystem.readFile({ path: file.path })
  const blob = new Blob([contents.data], { type: file.mimeType })
  formData.append("file", blob, file.name)

  const response = await CapacitorHttp.request({
    url: "https://example.com/upload",
    dataType: "file",
    data: formData,
    headers: {
      "Content-Type": "multipart/form-data",
    },
  })

  // ...
}

@kjr-lh
Copy link
Contributor Author

kjr-lh commented Jul 24, 2023

You can @jreviews, if you mean this:

import { CapacitorHttp } from "@capacitor/core"
import { Filesystem } from "@capacitor/filesystem"

export async function fileUpload(file: {
  name: string
  path: string
  mimeType: string
}) {
  const formData = new FormData()

  const contents = await Filesystem.readFile({ path: file.path })
  const blob = new Blob([contents.data], { type: file.mimeType })
  formData.append("file", blob, file.name)

  const response = await CapacitorHttp.request({
    url: "https://example.com/upload",
    dataType: "file",
    data: formData,
    headers: {
      "Content-Type": "multipart/form-data",
    },
  })

  // ...
}

I really don't expect this to work as-is - I think you'd need to have converted the form data already to a base 64 encoded string (take a look at what convertFormData is doing in native-bridge.ts

As it is now, I think capacitor will just try to convert the data to json, which means it will be sent to the native layer as "{}"

Now that the http plugin is part of capacitor core, I think the expectation is that everyone will just use fetch, as it's a more usual interface and code using it will also work on the web

@jreviews
Copy link

Thanks! I've released an update using fetch which is working fine, so I may come back and review this again later on.

@Mimateam
Copy link

You can @jreviews, if you mean this:

import { CapacitorHttp } from "@capacitor/core"
import { Filesystem } from "@capacitor/filesystem"

export async function fileUpload(file: {
  name: string
  path: string
  mimeType: string
}) {
  const formData = new FormData()

  const contents = await Filesystem.readFile({ path: file.path })
  const blob = new Blob([contents.data], { type: file.mimeType })
  formData.append("file", blob, file.name)

  const response = await CapacitorHttp.request({
    url: "https://example.com/upload",
    dataType: "file",
    data: formData,
    headers: {
      "Content-Type": "multipart/form-data",
    },
  })

  // ...
}

We also need to send a Blob through post method. We currently have mobile app env. with Ionic 7, Capacitor 5, Angular 16. In our case we need FormData, which we send it through post method to client's API. Client's API accepts only Blob.

We've tried angular/common/http which works great on dev env., but in production We get hit by client's CORS policy. So we need to use plugin which uses native libraries, like this one.

We would really appreciate it if there would be a way to send Blob directly through post with FormData. Thanks!

@jreviews
Copy link

@Mimateam The code I shared before #6708 (comment) using fetch works to send FormData with blob and it also makes request using native libraries. Just make sure you enable fetch patching per the capacitor http docs https://capacitorjs.com/docs/apis/http#configuration

@Mimateam
Copy link

We have now another problem. When using this approach #6708 (comment) then we cannot access images on mobile phone from cache when picking images from camera or gallery.

Solution to allow Blob directly through post with FormData would be highly appreciated and would solve all problems. We don't see why this should not be implemented?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants